{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ungraded Lab: Sigmoid function\n",
    "\n",
    "In this ungraded lab, you will implement the sigmoid function (also known as the logistic function).\n",
    "\n",
    "\n",
    "As discussed in the lecture videos, for a classification task, we can start by using our old linear regression model ($f(x) = w^Tx$) to predict `y` given `X`. \n",
    "- However, we'd like the predictions of our classification model to be between 0 and 1, since our output variable `y` is either 0 or 1. \n",
    "- This can be accomplished by using a \"sigmoid function\", which maps all input values to values between 0 and 1. \n",
    "\n",
    "\n",
    "Let's implement the sigmoid function and see this for ourselves.\n",
    "\n",
    "## Formula for Sigmoid function\n",
    "\n",
    "The formula for a sigmoid function is as follows -  \n",
    "\n",
    "$g(z) = \\frac{1}{1+e^{-z}}$\n",
    "\n",
    "In the case of logistic regression, `z` (the input to the sigmoid function), is the output of a linear regression model. \n",
    "- That is, `z` is not always a single number, but can also be an array of numbers. \n",
    "- If the input is an array of numbers, we'd like to apply the sigmoid function to each value in the input array. \n",
    "\n",
    "Let's implement this in Python.\n",
    "\n",
    "**Using numpy**\n",
    "\n",
    "First, let's run the cell below to import [numpy](www.numpy.org), which is the fundamental package for working with matrices in Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " `numpy` has a function called [`exp()`](https://numpy.org/doc/stable/reference/generated/numpy.exp.html), which offers a convinient way to calculate the exponential ( `e^{z}`) of all elements in the input array (`z`).\n",
    " \n",
    "It also works with a single number as an input, as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Input is an array. \n",
    "input_array = np.array([1,2,3])\n",
    "exp_array = np.exp(input_array)\n",
    "\n",
    "print(\"Input to exp:\", input_array)\n",
    "print(\"Output of exp:\", exp_array)\n",
    "\n",
    "# Input is a single number\n",
    "input_val = 1  \n",
    "exp_val = np.exp(input_val)\n",
    "\n",
    "print(\"Input to exp:\", input_val)\n",
    "print(\"Output of exp:\", exp_val)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the `exp` function, we can translate the sigmoid function formula $g(z) = \\frac{1}{1+e^{-z}}$ into code as follows: \n",
    "\n",
    "`g = 1/(1+np.exp(-z))`\n",
    "\n",
    "**Exercise** \n",
    "\n",
    "Now, use the code block shown above to implement the `sigmoid` function below in the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sigmoid(z):\n",
    "\n",
    "    ### START CODE HERE ### \n",
    "    g = \n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    return g"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see what the output of this function is for various value of `z`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generate an array of evenly spaced values between -10 and 10\n",
    "z = np.arange(-10,10)\n",
    "\n",
    "# Use the function implemented above to get the sigmoid values\n",
    "y = sigmoid(z)\n",
    "\n",
    "# Code for pretty printing the two arrays next to each other\n",
    "np.set_printoptions(precision=3) \n",
    "print(\"Input (z), Ouput (sigmoid(z))\")\n",
    "print(np.c_[z, y])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The values in the left column are the values in `z`, and the values in the right column are `sigmoid(z)`. As you can see, the input values to the sigmoid range from -10 to 10, and the output values should all range from 0 to 1. \n",
    "\n",
    "Now, let's try to plot this function using the `matplotlib` library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Plot z vs sigmoid(z)\n",
    "plt.plot(z, y, c=\"b\")\n",
    "\n",
    "# Set the title\n",
    "plt.title(\"Sigmoid function\")\n",
    "# Set the y-axis label\n",
    "plt.ylabel('sigmoid(z)')\n",
    "# Set the x-axis label\n",
    "plt.xlabel('z')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the sigmoid function asymptotes at `1` as `z` goes to large negative values and asymptotes at `0` as `z` goes to large positive values.\n",
    "\n",
    "Now try playing around with different input values to the sigmoid function, the output value should always be between `0` and `1`. You can also try using matrices as input to your sigmoid function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Try different input values. You can also try arrays as input\n",
    "sigmoid(90)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
